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Abstract 

There is well-documented evidence that online retention rates are lower than face-to-face retention rates. 
However, most past research on online retention focuses on student characteristics, with little knowledge 
existing on the impact of course type. This study uses a matched sample of 2,330 students at a large urban 
community college to analyze two key course-level factors which may be impacting online retention: the 
student’s reason for taking the course (as an elective or a requirement) and course difficulty level. The 
results of this study indicate that the online modality increases dropout risk in courses that are taken as an 
elective or distributional requirement, particularly for lower-level courses. The findings suggest that in 
the online environment, the student’s reason for course enrollment may be considered a risk indicator and 
that focused learner support targeted at particular course types may be needed to increase online 
persistence and retention. 


Introduction 

There has been a dramatic shift in higher education in the last decade toward online education. As 
a result, online courses are now a core feature of most community colleges and universities 
(Larreadmendy-Joems & Leinhardt, 2006; Layne, Boston & lee, 2013; Sutton & Nora, 2008). Today, 
more than 30% of all college students, and more than 60% of community colleges students, enroll in 
online courses. Online enrollments grew by one million students from 2009 to 2010, the largest ever 
year-to-year increase, far surpassing the growth of higher education in general, and have increased more 
than 29% since 2010 (Allen & Seaman, 2010; Community College Research Center, 2013; Pearson 
Foundation, 2011). With higher education enrollments exploding today, and more technologically savvy 
students seeking alternate pathways to higher education, online education is expected to keep growing in 
the years to come (Allen & Seaman, 2013). 

Concurrent with the accelerated growth in online education is escalating concerns about student 
outcomes (Boston & lee, 2011; Hachey, Wladis & Conway, 2013; Howell, Williams & Lindsay, 2003). 
Course completion serves as an important measure of both student outcomes and the success of an online 
program (Abel, 2005; Moody, 2004; Willging & Johnson, 2004). Student retention is a costly issue for 
institutions as well as their students. From the institutional perspective, student dropout incurs staff costs 
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to handle transitions, involves extra administrative and advising time, and results in lost revenue (Moody, 
2004). Moreover, reporting high completion rates is often critical for institutions to secure funding 
(Patterson & McFadden, 2009). From the student perspective, course dropout can impede progress 
towards a degree, cause financial loss, and incur potential psychological distress related to withdrawal 
decisions (Reed, 1981). 

Because of the high costs of student attrition and its association with program quality, there is a 
critical need for higher learning institutions to be able to predict the potential persistence of online 
students in order to direct targeted support towards ameliorating the problem (Hachey, Wladis & Conway, 
2013; Parker, 2003). So far, most research on student retention in the online environment focuses on 
student characteristics (Layne, Boston & lee, 2013; for reviews see Levy, 2007; Yukselturk & Bulut, 
2007); little knowledge exists on the impact of course type, even though it has been suggested as a 
potential online retention risk factor (Diaz, 2002). If course type is an online retention risk factor, then 
targeting particular courses for intervention, instead of seeking out individual students with particular 
characteristics, may be a more efficient and reliable way of allocating institutional resources to improve 
student persistence and online retention rates. We contend that the ability to predict online retention rates 
based on course characteristics may be extremely useful to institutions, since it could allow them to target 
with extra resources those particular course sections at highest risk of high online dropout—for example, 
including dedicated course-type specific counseling and coaching, providing course-specific online 
readiness surveys or special course orientations, and providing course-specific peer tutors. This study 
seeks to address the need for information about how course-level characteristics may impact online course 
retention by identifying and exploring two key factors: the student’s reason for taking the course (as an 
elective or to fulfill a distributional or major requirement); and course difficulty level. 


Background 

While several studies have found no significant difference between student learning online versus 
in traditional face-to-face classrooms (Bernard, et al., 2004; Russell, 2001), research nevertheless 
indicates a pervasiveness of non-completion in online learning. The online education literature 
consistently cites lower retention for online courses in comparison to face-to-face courses across higher 
education in the United States (Angelino, Williams & Natvig, 2007; Morris & Finnegan, 2008-9; Tyler- 
Smith, 2006)—retention in online learning programs is often reported as 7-20 percentage points lower 
than traditional campus-based programs (Hachey, Wladis & Conway, 2013; Moody, 2004; Nora & 
Plazas-Snyder, 2008; Patterson & McFadden, 2009; Smith & Ferguson, 2005). Based on extensive past 
findings, we expect to find retention rates to be lower for online courses in comparison to face-to-face 
courses. Although research has been conducted and findings reported of differences in student retention 
in online courses across fields (Finnegan, Morris & Lee, 2008-9; Neil, 2001), there seems to be little 
evidence in the online education literature looking at the impact of course level or a student’s reason for 
enrolling in the course (to fulfill elective, distributional or major requirements). Thus, what is not clear in 
the literature is if the risk of dropping out in the online modality is increased based on the type of online 
course that is taken. 

Many studies have considered how student characteristics affect course outcomes. However, this 
study takes a different approach—it focuses on differences at the course level. Colleges administering 
online programs are often looking for ways to target interventions at students who are at the greatest risk 
of poor online performance as compared to face-to-face performance. One way of targeting those 
students is to use student characteristics; however, this method is time consuming and difficult because it 
requires colleges to collect and track information on a number of student characteristics which are not 
routinely collected (e.g. motivation, work hours), and because it requires colleges to target students 
individually. This study seeks to explore if there are particular courses which may have lower successful 
online completion rates than would be expected given successful face-to-face completion rates of the 
same course (taught by the same instructor). If such courses can be identified, then colleges can target 
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interventions to those courses at greatest risk, and in this way, target the students at greatest risk. The 
reason why these courses are at a greater risk online could be a number of things: it could be that the 
characteristics of students who take these courses make them particularly at-risk in the online 
environment, or it could be that there are characteristics of the courses themselves that make them more 
poorly suited to the online environment. The aim of this study is to identify which courses are at risk so 
that resources could be targeted to these classes; the goal of future work would, thus, be to explore the 
reasons behind any differences which are uncovered by this study. 

Course Difficulty Level 

In this paper, we define course difficulty as the level of a course, or whether a course requires 
credit-bearing prerequisites. For the college in this study, lower level courses are 100-level courses which 
do not have prerequisites (other than possible developmental coursework); in contrast, 200-level courses 
and above require at least one 100-level course as a prerequisite. In this way, 200-level courses cover 
more advanced material, which is why we refer to them as having a higher difficulty level. There are, of 
course, other ways in which the difficulty level of a course could be interpreted, but we do not purport to 
cover all of those interpretations here. Our goal is simply to distinguish between courses which have 
credit-bearing pre-requisites versus those that do not. Our reasons for focusing on this particular 
distinction are based on some evidence in the literature that students in lower-level courses may be more 
vulnerable to doing worse online than would be expected given their face-to-face performance. 

Two recent studies of community college students found that students who took online classes 
early in their college careers were more likely to drop out than those who took only face-to-face courses 
(Jaggars & Xu, 2010; Xu & Jaggars, 2011), which might suggest that students enrolled in lower level 
classes which are typically taken earlier in a college career might be at greater risk of dropping out. Since 
difficulty of instructional materials has been cited as one potential reason students drop out of online 
courses (Diaz, 2002), the level of difficulty of a course may be one factor that may make it more prone to 
higher attrition in the online environment. Some prior research has found a strong negative correlation 
between previous education in the discipline and dropping out of an online course. This indicates that 
students may be more likely to drop out of lower level online courses, particularly when they are in a 
subject unrelated to their prior course experience. In other words, students may be more likely to drop 
lower level online courses that are outside their major (Xenos, Pierrakeas & Pintelas, 2002). 

Reason for Course Enrollment (to fulfill Elective, Distributional or Major Requirements) 

In particular, there seems to be little research that looks at whether the decision to enroll in 
required versus elective online courses has an impact on online retention. Student perceptions of online 
learning have been shown to be a better predictor of outcomes at the post-secondary level than grade 
point average (G.P.A.) (Lizzio, Wilson & Simons, 2002; Sutton & Nora, 2008). Moreover, across most 
disciplines, whether a course is an elective or a requirement is a variable that has been linked to student 
attitudes (Babad, 2001). Consistently in the face-to-face course literature, elective courses receive better 
evaluations than required courses (for a review, see Darby, 2006). Given the influence of student 
perception on persistence, this suggests that retention for elective courses may be higher than required 
courses. However, Reed (1981) contends that persistence in a course is significantly related specifically 
to students’ belief in the relevance of the course to their need and, further, found that students were much 
more likely to drop courses that were electives. 

Rational choice theory may provide an additional framework, beyond that of student persistence, 
for examining student course selection. Rational choice theory posits that individuals will base their 
activities and decisions on a cost-benefit analysis (Coleman & Fararo, 1992). If students believe there is a 
greater “pay off’ from a specific course or set of courses, it may induce the student to persist. Students 
might consider there to be a greater “payoff’ when selecting and completing courses that are required for 
their degree versus electives, and may therefore be more likely to persist in required courses. 
Additionally, to the extent that students perceive courses in the online environment to be less rigorous, 
this perception might be an inducement for enrollment. However, such perceptions may also lead to 


3 



The Role of Enrollment Choice in Online Education 


greater attrition when students find the course to be more difficult than originally thought (Moody, 2004). 
Course workload and course difficulty are oft cited reasons for course withdrawal but often not the 
primary reason (Babad, leekson & Yelenik, 2008; Summer & Johnson Community College, 2001). In a 
survey of 500 undergraduates enrolled in face to face courses, Babad & Tayeb (as cited in Babad, leekson 
& Yelinek, 2008) found that students primarily dropped a course due to “atmosphere”, a factor that was a 
composite of both a student’s connection with classmates (the number of students, the quality of the 
discussion) and physical conditions (classroom crowding and the characteristics of the room itself). But 
as Babad & Tayeb (as cited in Babad, leekson & Yelinek, 2008) assert, the components of “atmosphere” 
used in this study may be more nebulous in the online environment. 

At the college in this study, an interesting pattern was observed during a preliminary review of 
retention rates that seemed to suggest that the online environment may affect the retention rate more 
strongly for students taking courses as a distributional versus a major requirement. Retention rates in two 
different mathematics courses taught by the same professor, in the same semester, and with the same 
prerequisites were significantly different for the online section of the course as compared to the face-to- 
face section. Fundamentals of Mathematics (MAT 100) is taken by liberal arts majors as one option for 
fulfilling their mathematics distributional requirement. Mathematics for Health Sciences (MAT 104) is a 
required course in certain health care majors such as paramedics and nursing, with the course content 
entirely focused on applied problems in the field. In this example, the attrition rate in the online section 
of MAT 100 (the distribution requirement) was almost double that of its face-to-face counterpart; whereas 
the attrition rate in MAT 104 (the major requirement) was slightly (but not significantly) lower in the 
online environment compared to the face-to-face version. These preliminary findings suggest that an 
exploration of course-level factors, such as a student’s reason for taking a course (to fulfill elective, 
distributional, or major requirements), is a logical next step in building predictive models of online 
student persistence. If course-level factors can be identified which allow institutions to identify courses 
which are at risk of significantly higher attrition rates in the online environment than would be expected 
given attrition rates in face-to-face environments, then interventions can be targeted to these courses. The 
reason why some courses may be at a greater risk in their online form could be that the characteristics of 
students who take these courses make them particularly at-risk in the online environment; however, 
focusing interventions on students with specific characteristics is more resource-intensive and difficult 
than targeting particular types of courses. Therefore, this study focuses on characteristics which are at the 
course rather than student level. 


Purpose of the Study 

The purpose of this study is to assess the extent to which students’ reasons for taking a course (to 
fulfill elective, distributional, or major requirements) and course difficulty level may be used as predictors 
of online versus face-to-face course outcomes. Therefore, the following questions are addressed: 

Which combinations of course-level factors (elective vs. distributional vs. major requirements; 
level) have retention rates that are significantly lower online than face-to-face? 

When comparing online and face-to-face sections of the same course, is the gap in retention rates 
larger for courses which students take as elective or distributional requirements than for courses which 
students take to fulfill major requirements? 

When comparing online and face-to-face sections of the same course, is the gap in retention rates 
larger for lower-level (100-level) courses than for upper-level (200-level and above) courses? 

Is there an interaction between a student’s reason for taking a course (to fulfill elective, 
distributional or major requirements) and course difficulty level in predicting online versus face-to-face 
course retention? For example, is the correlation between a student’s reason for taking a course and 
online course outcomes greater for lower-level courses than for upper-level courses? 
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Methodology 

Data Source and Sample 

This study utilizes data provided by the Office of Institutional Research from a large, urban 
community college in the Northeast region of the United States. Enrolling approximately 23,500 students 
in degree-programs each year, the college meets the requirements to be deemed a “large institution”, 
according to Allen & Seaman (2010). This is noteworthy as large institutions educate nearly two-thirds 
(64%) of all online students. The gathered data represents a diverse student body, as enrollees come from 
over 150 countries around the world. Eighty percent of the College’s student population belongs to 
groups historically underrepresented in higher education: the College is classified as both a Minority 
Serving Institution and a Elispanic Serving Institution. The College has offered online classes since 2002, 
and currently offers a fully online Associate’s Degree in Liberal Arts in addition to over 135 online 
courses. 

Data was compiled for 122 course sections (half taught online and half taught face-to-face). The 
course sections to be included in the sample were selected from a larger pool using the following 
methods: first, data were obtained for all online course sections taught from 2004-2010 (fall and spring 
semesters). Second, the sample was limited to include only those course sections for which an instructor 
taught the same course both face-to-face and online in the same semester, to control for instructor effects. 
Next, the sample was reduced to only those courses which the instructor had taught for at least three 
semesters, to limit the effects of instructor inexperience in the online environment. In total, 21 different 
courses taught by 23 different instructors both online and face-to-face were included. The courses were 
well distributed across disciplines, with three in business, one in nursing, five in the humanities, five in 
the social sciences, four in mathematics, and three in science disciplines. 

Measures 

For each section in the sample, student data was provided stripped of personal identifiers; unique 
identification numbers were used to distinguish participants. This resulted in a total dataset of 2,330 
participants. The following information was obtained for every student: student major (used to determine 
if the course was taken as a requirement or elective) and final grade in the course (including withdrawal 
status). The categorization of a course as an elective, distributional requirement, or major requirement for 
each student was determined based on the requirements of students’ majors: courses which did not fulfill 
any particular curriculum requirement (other than general elective credits) were counted as electives; 
courses that fulfilled a degree requirement not part of the major’s core curriculum were counted as 
distributional requirements; and courses that were either explicitly required as a part of a major’s core 
curriculum, or were elective major courses that counted as major requirements for a particular student 
were coded as major requirements. Major requirements were not always limited to courses in the exact 
subject of the student’s major. Courses in related fields were also counted as major requirements if they 
were listed by a department as being required for the major in the college catalog. 

The dependent variable which was the focus of this study was retention: students were 
categorized as having completed the course if they were still attending class after the tenth week of the 
semester (only fall and spring semesters were included in the analysis). Students who never attended the 
course or who received an incomplete grade were excluded from the analysis. 

Data Analyses 

First we performed significance tests using the Bonferroni procedure and z-scores to determine 
whether different types of courses in the sample had significantly different retention rates (online versus 
face-to-face; lower-level versus upper-level; elective versus distribution versus major requirements). 
Then we used these same types of significance tests to determine whether each course subtype (lower 
level; upper level; elective; distributional requirement; major requirement) had significantly lower 
retention rates in their online form versus their face-to-face form. Based on current research literature, we 
expected (but did not assume) average retention rates to be lower online than in face-to-face sections. 
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Elowever, because the existence, direction, and size of the gap between online and face-to-face 
completion rates may vary by course type, we used binary logistic regression to determine if any 
differences in the size or direction of retention rate gaps between different course types (elective, 
distributional, major requirement) were significant by interacting course-level characteristics with the 
course medium (online versus face-to-face). 

We performed significance tests using the Bonferroni procedure and z-scores to determine 
whether online and face-to-face retention rates were significantly different for combinations of subtypes 
of courses (e.g. lower level electives). We then conducted separate binary logistic regression models for 
lower level courses and for upper level courses that compared the differences in size and direction for the 
gap between online and face-to-face courses among elective versus distributional versus major 
requirements. 

Results and Discussion 

Preliminary Analysis of the Data 

As a precursor to analyzing differences in retention in different online course types, we first 
analyzed whether, in this sample, there were generally (not just online) differences in attrition rates 
among courses taken to fulfill elective versus distributional versus major requirements, or among upper 
versus lower level courses. Pooling all students in the sample in both online and face-to-face courses to 
compute retention rates for each category yields the data in Table 1, which includes tests for significance. 
(We note that Retention rates and Attrition rates are complements of one another: the Attrition rate is the 
percentage of students who withdrew from the course officially or unofficially [i.e. a student stopped 
attending class sometime before the tenth week of the semester—students who stop attending after the 
10th week of classes receive an “F” grade instead], whereas the Retention rate is the percentage of 
students who did not withdraw officially or unofficially, but may have earned an “F” or “D” grade.) 


Table 1 Retention rates for students in each classification of course type, with tests for significance. 



retention 

n 

z 

P 

z-score compares: 

face-to-face 

81.0% 

1107 

5.46 

<0.0001 

face-to-face vs. online 

online 

70.6% 

887 




lower level 

69.3% 

1092 

-8.16 

<0.0001 

lower level vs. upper level 

upper level 

84.9% 

902 




elective 

67.7% 

449 

-2.79 

ns 

elective vs. dist. req. 

dist. req. 

74.8% 

980 

-5.21 

<0.0001 

dist. req. vs. major req. 

major req. 

86.0% 

565 

6.98 

<0.0001 

elective vs. major req. 


Results for p-values in bold are all highly statistically significant (a=0.01, two-tailed), even when the Bonferroni 
procedure is used to control for Type 1 error. The abbreviation ns means that the result is not statistically 
significant. 


In Table 1 it is apparent that attrition is higher: 1) in online courses compared to face-to-face 
courses; 2) in lower level courses compared to upper level courses; and 3) in courses that are elective or 
distributional requirements rather than major requirements. All of these differences are highly 
statistically significant (a=0.01). 

Differences in online versus face-to-face course retention rates for different combinations of course- 

level factors 

Next, we considered the interaction between the method of course delivery (online vs. face-to- 
face) with course type (elective vs. distributional requirement vs. major requirement) and level. Retention 
rates for each course-level factor can be seen in Table 2 broken down by course delivery modality. 
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Table 2 Retention rates online and face-to-face for different course types, with 

tests for significance 



face-to-face 

retention 

n 

online 

retention 

n 

z 

P 

lower level 

75.5% 

593 

61.9% 

499 

4.86** 

<0.0001 

upper level 

87.4% 

514 

81.7% 

388 

2.35 

ns 

elective 

77.8% 

203 

59.3% 

246 

4.17** 

<0.0001 

dist. req. 

79.1% 

560 

69.0% 

420 

3.59** 

<0.0001 

major req. 

86.0% 

344 

86.0% 

221 

0.02 

ns 


** indicates significance level of a=0.01 (one-tailed) for overall set of tests 
(adjusted to 0.0011 per test using the Bonferroni procedure) 


In Table 2, retention followed the same pattern in the online environment as in the face to face 
environment: retention was greater in upper level and major requirement courses versus lower level and 
elective or distributional requirement courses. Additionally, the following course types had a highly 
statistically significantly (a=0.01) lower retention rate online than face-to-face: lower level courses; 
elective courses, and courses that fulfill distributional requirements. This suggests that whatever factors 
lead to lower overall retention rates for lower level, elective, and distributional requirement courses (e.g. 
lower levels of student motivation) may be amplified by the online environment. The differences in 
online versus face-to-face course outcomes by course type are illustrated graphically in Figure 1. 


Figure 1 Online and Face-to-Face Retention for Electives vs. Distributional Requirements vs. Major Requirements 

l 

0.9 
0.8 
0.7 
C 0.6 

o 
*5 

C 0.5 

0 ) 

+■* 

QJ 

i- 0.4 
0.3 
0.2 
0.1 
0 

face-to-face online 

course delivery method 



It is evident from Figure 1 that students who take an online course which fulfills their major 
requirements are roughly equally as likely to remain in the course whether they take it online or face-to- 
face, while students who take the course to fulfill a distributional requirement or as an elective are much 
more likely to withdraw online than in the face-to-face environment, with this difference particularly 
pronounced for electives. To determine if the slopes of the lines in Figure 1 are statistically significant, 
we performed a binary logistic regression with retention as the dependent variable and course delivery 
method and course type as the independent variables. We also included the interaction between these two 
factors as a term in the regression equation. The results of this analysis can be seen in Table 3. 
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Table 3 Type III analysis for binary logistic regression analysis of the interaction effect of course delivery method and 

course type on retention 


Chi-square (LR) 

Pr > LR 

course delivery method 

0.037 

0.847 

course type 

7.328 

0.026* 

course delivery method*course type 

8.661 

0.013* 

-2 Log Likelihood 

2376.297 


R 2 (Nagelkerke) 

0.053 


* indicates significance level of a=0.05 


When the interaction between course delivery method and course type is taken into account, the 
interaction between course type and course delivery method is statistically significant (a=0.05). In other 
words, the differences in slope that we see in the lines in Figure 1 are statistically significant, and students 
in elective and distributional requirement courses have a much higher jump in attrition when they move to 
the online environment than students in major requirement courses. 

But what happens if course type and level are combined? For example, would an upper level 
elective course have lower online retention or not? To address these questions, we next analyzed 
differences in retention rates for the two factors of course type and level, both online and face-to-face. 

How does course level interact with course type and the online environment in predicting course 

outcomes? 

First we looked at each subgroup of course level by type (e.g. lower level elective courses, upper 
level distributional requirements, etc.), to see which of these groups had retention rates that were lower in 
the online environment. This data is displayed in Table 4. 


Table 4 Retention rates online and face-to-face for different combinations of course type, with tests for 

significance 


level 

type 

face-to-face 

retention 

n 

online 

retention 

n 

z 

P 

lower level 

elective 

74.30% 

105 

52.10% 

165 

3.64 

<0.0001** 

lower level 

dist. req. 

75.90% 

390 

64.50% 

290 

3.24 

0.0006** 

lower level 

major req. 

75.50% 

98 

81.80% 

44 

-0.83 

ns 

upper level 

elective 

81.60% 

98 

74.10% 

81 

1.21 

ns 

upper level 

dist. req. 

86.50% 

170 

79.20% 

130 

1.68 

ns 

upper level 

major req. 

90.20% 

246 

87.00% 

177 

1.03 

ns 


* indicates a significance level of a=0.01 (one-tailed) for overall set of tests (adjusted to 0.0017 
per test using the Bonferroni procedure) 


In Table 4, it is apparent that lower level courses taken as either electives or to fulfill 
distributional requirements have statistically significantly (a=0.01) lower retention rates online than face- 
to-face, whereas this effect is not seen for major requirements or for upper level courses. In Figure 2 and 
Figure 3, these differences are displayed graphically. 

For lower level courses (Figure 2), all course designation types (electives, distributional 
requirements and major requirements) have almost identical retention rates face-to-face. In the online 
environment, however, these retention rates greatly diverge, covering a range that is about thirty 
percentage points wide. For lower level courses in the online environment, major requirement courses 
have retention rates that are actually a bit higher than in the face-to-face environment (although this 
difference is not statistically significant), whereas distributional requirement courses have retention rates 
that are significantly lower online, and elective courses have retention rates that are much significantly 
lower online. In contrast, for upper level courses, major requirement courses have higher retention than 
distributional requirement courses, which have higher retention than elective courses, but this pattern is 
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nearly identical both online and face-to-face: the gap between courses is about the same both face-to-face 
and online, and the retention rates for each category, while slightly lower online, are not statistically 
different from the retention rates for each corresponding category face-to-face. 


Figure 2 Retention for Lower Level (LL) 
courses by type and delivery method 


Figure 3 Retention for Upper Level (UL) 
courses by type and delivery method 



1 

0.9 -- 
0.8 -- 
0.7 
0.6 -- 


C 0.5 

0 

V 0.4 
0.3 -- 
0.2 
0.1 
0 


- - * 

•—__ __J~ ~ ~-- 


UL major req. 


—•- - UL dist. req. 
— «— UL elective 


-+- 


face-to-face online 

course delivery method 


To determine whether or not the differences in slopes in Figure 2 and Figure 3 are statistically 
significant, we performed a binary logistic regression—first on lower level courses only, and then on 
upper level courses only—to determine what effect the interaction of course delivery method and course 
type might have on retention. These results can be seen in Table 5. 


Table 5 Type III analysis for binary logistic regression analyzing the effects of the interaction between course delivery 
method and course type on retention, broken down by course level 

lower level courses 

Chi-square (LR) 

Pr> LR 

course delivery method 

0.226 

0.634 

course type 

0.096 

0.953 

course delivery method*course type 

7.356 

0.025* 

-2 Log Likelihood 

1556.1 


R 2 (Nagelkerke) 

0.042 


upper level courses 

Chi-square (LR) 

Pr> LR 

course delivery method 

6.889 

0.009** 

course type 

12.385 

0.002** 

-2 Log(Likelihood) 

769.0 


R 2 (Nagelkerke) 

0.037 


** and * indicate significance levels of of 

=0.01 and a=0.05 respectively 



In Table 5, we can see that once the interaction between course delivery method and course type 
is taken into account in the lower level course subsample, the interaction is statistically significant. So the 
differences in slope that are visible in Figure 2 are in fact statistically significant, and knowing the type or 
delivery modality of a course alone does not provide us particularly useful information for predicting 
online course outcomes in lower level courses. Instead the differences in retention rates among these 
different categories is better captured by knowing the course type and modality subcategory (for example, 
knowing that a course is online, or is an elective is not enough to predict a retention rate that might be 
different from average for lower level courses, but knowing that a course is an elective online is 
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predictive of retention). 

Table 5 also gives the results of a binary logistic regression analysis on upper level courses. In 
this model the interaction term was omitted because it was not significant, and both course modality and 
type are significant predictors of retention, which is what we would expect from looking at the graph. For 
upper level courses, online retention rates are slightly but significantly lower, and the retention rates for 
major requirements are higher than for distributional requirements, which are in turn higher than for 
electives, in both the online and face-to-face modality. Planned pairwise comparisons during the binary 
logistic regression analysis (not shown here for the sake of brevity) show that distributional requirements 
and electives had statistically lower retention rates overall, but that the difference between elective and 
distributional requirements was not significant. This means that for upper level courses, unlike for lower 
level courses, the effects of the online environment on retention seem to be roughly the same across all 
course types. 

Limitations 

This research was limited to a specific sample chosen from a single community college, and as 
such, these results may not necessarily generalize nationally or to four-year institutions. However, this 
limitation is mitigated in several important ways. First, the community college in this study has a very 
diverse student body. Second, because the study focuses on a large urban community college, the sample 
the data was drawn from is representative of the types of institutions which educate the vast majority of 
community college students in the United States, since 82% of all community college students attend 
institutions in or on the fringe of mid- and large-sized cities (U.S. Department of Education, Institute of 
Education Sciences, National Center for Education Statistics, 2003). This suggests that research based on 
the college in this study may be relevant to a large proportion of community college students nationally. 
Finally, studying students within a single institution rather than across institutions limits threats to internal 
validity, because faculty conditions, course requirements and institutional elements are more uniform 
within a single institution (Nora & Cabrera, 1996). 

In addition, controlling courses by instructor and semester (including in the sample only pairs of 
course sections taught by the same instructor both online and face-to-face in the same semester) limits the 
total number of possible instructors included in the sample, and therefore each instructor may have greater 
influence on the results. Because all the instructors in this sample taught online for at least three 
semesters (many of them much longer) and because the college in this study does not let faculty continue 
to teach a course online if students in their courses have very low retention rates, it is unlikely that a small 
number of faculty with particularly low retention rates strongly affected the results. With that said, it 
would be necessary to repeat this study with larger sample sizes at other institutions to confirm the 
general trends observed here. 

Additionally, the differences in significance for factors in the models included in Table 5 for 
lower level courses versus upper level courses is not in and of itself proof of a three-way interaction 
between course type, level, and delivery method. Technically only a larger binary logistic regression 
model containing all two-way and three-way interactions could ascertain the statistical significance of the 
differences between the trends seen in Figure 2 for lower level courses versus those seen in Figure 3 for 
upper level classes. However, the sample size in this particular study was not sufficiently large to allow 
for the necessary statistical power to perform such a test. As such, the differences observed between the 
patterns seen in Figure 2 and Figure 3 should be interpreted as preliminary results only, which should be 
tested with larger samples. From the institutional perspective, the results of such a statistical test may be 
less relevant: to improve online retention rates, it may not be necessary to determine for certain whether 
the patterns observed for upper level courses in Figure 3 are truly significantly different from those 
observed for lower level courses in Figure 2. Rather, the simple fact that lower level courses taken as 
electives or distributional requirements have the highest drop in retention when moved to the online 
environment suggests that targeting this specific subgroup of courses for extra support in the online 
environment may be sufficient to improve overall online attrition. 
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Lastly, this research did not explore differences in student characteristics. Numerous studies, 
including our own (e.g. Jaggars & Xu, 2010; Wladis, Elachey, & Conway, in press; Xu & Jaggars, 2011), 
have shown that students who possess certain non-traditional characteristics, including being older, 
working, attending part-time and supporting dependents, are more likely to enroll in online courses. 
These same characteristics are also factors which have been shown to negatively impact persistence, 
regardless of course modality. However, the goal of this research is not to compare outcomes of online 
versus face-to-face students directly. Rather, the goal of this study is to look at the interaction between 
the online medium and course characteristics, which focuses instead on exploring when the gap between 
online and face-to-face completion rates is significantly larger or smaller for different types of courses. 

It is possible that certain types of courses attract students who are more likely to be at risk in 
online environments, and this could explain any significant differences in the online-versus-face-to-face 
course completion gap for specific course types. Our study does not discount this as a possible reason for 
differences in outcomes for different types of courses. For example, students who are enrolled in more 
advanced level courses have already succeeded to an extent that has not (and may not) occur for students 
enrolled in lower level courses, and this may explain the differences in the online-versus-face-to-face 
course completion gap for higher versus lower level courses. Whatever the reason for larger gaps in 
specific types of courses, identifying courses with larger gaps could allow institutions to target 
interventions specifically to those course types. Identifying courses that are at highest risk in the online 
environment allows resources to be targeted to these classes, and then the goal of future work would be to 
explore the reasons behind differences which have been uncovered by this study. 


Implications 

For Practice 

Ludwig-Hardman & Dunlap (2003) contend that individualization in learner support services is 
greatly needed to increase retention in online courses. This research suggests that online course retention 
rates can be improved by providing extra support targeted specifically to lower level courses which are 
typically taken as electives or to satisfy distributional requirements. Such support could include self- 
assessment and orientation tools which could be used to help students assess their perceptions and 
preparedness for the course. At the course level, E-advisors could provide an early mechanism for 
academic counseling, additional technical support staff could assist students with technical difficulties 
specific to the online environment, and peer tutors could assist students with the course content. Such 
measures could improve retention rates for these courses where students are at highest risk of dropping 
out. Retention online overall could be improved just by targeting the subgroups of courses at highest risk 
(thereby more efficiently allocating resources). In particular, for the sample in this study, if an 
intervention (effective enough to improve retention online to the level typical in comparable face-to-face 
courses) was targeted only at lower level elective and distributional requirement courses, the overall 
online persistence rate would have jumped from 70.6% to 78.4%, and this increase in overall online 
course persistence would be sufficient to close the gap with overall face-to-face persistence rates (80.0%). 
The 78.4% online persistence rate that could be obtained by such an intervention is not significantly 
different from the 80.0% face-to-face persistence rate (at the a=0.05 level with a one-tailed z-test). This 
suggests that if a sufficiently effective intervention could be implemented, any online and face-to-face 
attrition rate gap could be closed by targeting just roughly half of the online courses offered at this 
college. 

Some institutions might conclude from these results that they should limit or prohibit certain 
types of courses to be taught online, because those courses have a lower retention rate online than face-to- 
face. While this might raise average online retention rates, in practice it would be impractical by limiting 
access for a huge number of students, and would be a misapplication of the conclusions in this analysis. 
It is important to note that the designation as an elective or distributional requirement lies with the 
student, not the course, as it depends upon a student’s major. For example, English 201 may be taken at 
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the college in this study by many students as a distributional requirement, but for English or Writing 
majors, this course actually fulfills their major requirements. Even if it were practical, removing all the 
students from online courses who took the course as an elective or a distributional requirement would 
require prohibiting more than half of the online students from taking courses online, which would defeat 
the purpose of online education in providing greater access to higher education. In particular, the many 
students who do succeed in these online courses would have fewer course options and, therefore, might be 
less likely to persist in college and complete college degrees. Furthermore, removing certain types of 
courses from the online environment may not actually improve overall online retention. For example, it 
may be that lower level online courses have lower retention because they contain a much higher 
proportion of students taking an online course for the first time. If this is the case, eliminating lower level 
online courses may then simply shift increased drop-out rates from lower level to upper level courses. 

For Research 

This study has shown that the type of course in which students enroll can have a drastic effect on 
their likelihood of withdrawal from online courses. Elowever, before larger generalizations can be made 
about which types of courses lead to lower retention online in the general college student population, this 
analysis should be repeated with diverse samples across different campuses. In addition, the reasons for 
the lower rates of retention in lower level online classes taken as electives and distributional requirements 
are unclear, and further research could help to explain the reasons for these results. 
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